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Abstract 

Compressive  sensing  (CS)  is  a  new  signal  acquisition  technique  for  sparse 
and  compressible  signals.  Rather  than  uniformly  sampling  the  signal,  CS  com¬ 
putes  inner  products  with  randomized  basis  functions;  the  signal  is  then  re¬ 
covered  by  a  convex  optimization.  Random  CS  measurements  are  universal 
in  the  sense  that  the  same  acquisition  system  is  sufficient  for  signals  sparse  in 
any  representation.  This  paper  examines  the  effect  of  quantization  of  CS  mea¬ 
surements.  A  careful  study  of  stictly  sparse,  power-limited  signals  concludes 
that  CS  with  scalar  quantization  does  not  use  its  allocated  rate  efficiently.  The 
inefficiency,  which  is  quantified,  can  be  interpreted  as  the  price  that  must  be 
paid  for  the  universality  of  the  encoding  system.  The  results  in  this  paper  com¬ 
plement  and  extend  recent  results  on  the  quantization  of  compressive  sensing 
measurements  of  compressible  signals. 

1  Introduction 

Sparse  representations  have  received  significant  attention  in  the  signal  processing  lit¬ 
erature.  Their  importance  is  highlighted  in  applications  to  transform  coding  and 
signal  compression.  In  these  applications,  the  signal  is  acquired  using  a  high-rate 
sampling  process.  The  samples  are  subsequently  transformed  to  a  sparse  linear  rep¬ 
resentation,  in  which  most  of  the  representation  coefficients  are  zero  or  close  to  zero. 
This  sparsity  is  exploited  in  the  encoding  process  to  reduce  the  rate  required  to  store 
and  transmit  the  signal. 

Recent  results  in  the  area  of  compressive  sensing  demonstrate  that  signals  with 
sparse  or  approximately  sparse  representations  can  also  be  efficiently  acquired  [1-4]. 
Classical  sampling  theory  suggests  that  sampling  and  reconstruction  of  a  signal  in 
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a  vector  space  requires  at  least  as  many  linear  measurements  of  the  signal  as  the 
dimensionality  of  the  space.  Compressive  sensing  significantly  reduces  the  number  of 
measurements  required  to  capture  and  subsequently  reconstruct  the  signal,  as  long 
as  it  has  a  sparse  representation  or  can  be  well  approximated  by  one.  The  results 
also  guarantee  that  the  signals  can  be  recovered  sufficiently  well,  even  in  the  presence 
of  measurement  error  and  distortion  such  as  quantization.  One  of  the  advantages  of 
compressive  sensing  is  the  universality  property,  which  guarantees  that  with  proper 
random  selection  of  the  measurement  space,  the  signal  can  be  recovered  with  very 
high  probability,  even  if  the  representation  in  which  the  signal  is  sparse  is  not  known 
in  advance  of  the  measurements  [2], 

The  sparsity  models  used  in  the  compressive  sensing  literature  vary,  depending 
on  the  application.  A  sparsity  dictionary  is  always  assumed,  which  is  a  collection 
of  vectors — usually  an  orthonormal  basis — in  which  the  signal  can  be  sparsely  rep¬ 
resented.  One  common  signal  model  assumes  that  the  signal  is  power  limited  and 
sparse,  i.e.  has  very  few  non-zero  coefficients  when  represented  using  the  dictionary. 
Other  common  models  assume  that  the  representation  is  compressible,  i.e.  that  very 
few  coefficients  are  sufficient  to  approximate  the  signal  well  enough.  The  most  com¬ 
mon  model  for  compressible  signals  assumes  that  the  representation  lies  in  an  £p  ball, 
i.e.  has  bounded  £p  norm  for  some  p  <  1.  Under  this  assumption,  which  guarantees 
compressibility,  the  quality  of  the  approximation  depends  on  p  and  the  number  of 
representation  coefficients  used  to  approximate  the  signal. 

The  quantization  of  compressible  signals  with  bounded  £p  norm  in  the  context  of 
compressive  sensing  has  been  studied  in  [5] .  Specifically,  lower  bounds  for  the  quanti¬ 
zation  error  are  presented,  applicable  to  any  quantized  representation.  These  bounds 
are  compared  with  the  performance  of  compressive  sensing  followed  by  scalar  quanti¬ 
zation  and  non-linear  reconstruction.  It  is  thus  demonstrated  that  the  performance  of 
a  compressive  sensing  universal  encoding  method  is  within  a  logarithmic  factor  of  the 
optimal  encoding.  This  is  a  small  price  to  pay  for  the  universality  of  the  algorithm. 

The  main  contribution  of  this  paper  is  to  examine  the  quantization  of  strictly 
sparse,  power  limited  signals  in  terms  of  the  quantization  error  and  the  rate-use  effi¬ 
ciency  of  compressive  sensing.  Although  the  results  presented  extend  and  complement 
the  work  on  the  quantization  error  presented  in  [5],  sparse  signals  behave  very  dif¬ 
ferently  than  compressible  signals  with  bounded  £p  norm.  The  strictly  sparse  signal 
model  is  better  suited  to  describe  man-made  signals,  such  as  radar  and  communi¬ 
cation  signals.  It  is  also  a  good  model  for  signals  that  have  already  been  encoded 
and  subsequently  decoded  using  classical  transform  coding  methods,  and  have  thus 
become  sparse  in  the  process. 

After  presenting  some  background  and  establishing  the  notation  in  Sec.  2,  it  is 
shown  in  Sec.  3.1  that  compressive  sensing  measurements  of  sparse  signals  use  only 
a  fraction  of  the  available  quantization  points  to  encode  the  signal.  This  implies 
that  the  encoding  bits  are  inefficiently  used.  This  inefficiency,  which  is  examined 
and  quantified  in  Sec.  3.2,  can  be  considered  as  the  cost  to  achieve  the  universality 
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of  compressive  sensing  encoding,  or  a  target  for  elimination  by  subsequent  lossless 
entropy  coding. 

Using  the  bounds  on  the  quantization  efficiency,  a  lower  bound  for  the  quantization 
error  is  derived  in  Sec.  3.3.  It  thus  follows  that  compressive  sensing  and  quantization 
of  sparse  signals  incur  a  penalty  in  the  error,  compared  to  the  optimal  error  achievable 
using  transform  coding.  This  penalty  can  also  be  considered  as  the  cost  of  universality. 
It  is  also  shown  that  reconstruction  using  linear  programming  incurs  a  further  penalty 
in  the  achievable  error.  The  results  in  this  paper  are  used  in  Sec.  3.4  to  improve  the 
lower  bounds  presented  in  [5],  specifically  in  the  case  of  low  rate  quantization  and 
highly  compressible  signals. 

2  Background 

This  section  presents  some  background  on  sampling,  quantization,  and  compressive 
sensing.  The  main  purpose  is  to  establish  the  notation  and  serve  as  a  quick  reference. 

2.1  Sampling  and  Quantization 

For  the  purposes  of  this  paper  we  assume  a  signal,  denoted  using  x,  that  lies  in  an 
N  dimensional  space,  denoted  using  S,  and  referred  to  as  the  signal  space.  Sampling 
is  performed  by  taking  measurements  using  inner  products  of  x  with  an  orthonormal 
sampling  basis  {4>k\k  —  1  The  sampling  basis  spans  the  M-dimensional 

sampling  space  W,  which  is  a  subspace  of  S.  The  sampling  operator  is  defined  as: 

$  :  S  ->  Mm,  s.t.  ($x)fc  =  (x,0fe),  (1) 

in  which  (•)*,  is  the  kth  component  of  a  vector  in  MA/.  The  vector  of  sampling  co¬ 
efficients  is  denoted  using  y  =  <t»x.  Correspondingly,  the  reconstruction  function  is 
denoted  using: 

R(-),  s.t  R  :  Rm  — >  S.  (2) 

The  parenthetical  notation  is  used  as  a  reminder  that  the  reconstruction  is  not  nec¬ 
essarily  linear. 

In  order  to  process  and  transmit  the  sampling  coefficients,  they  are  quantized  to  a 
finite  precision.  It  is  assumed  that  each  coefficient  is  mapped  to  one  of  L  quantization 
levels,  in  which  L  is  finite.  Unless  otherwise  noted,  most  of  the  results  require  no 
assumption  on  the  quantization  intervals.  In  some  cases  uniform  quantization  is 
assumed  and  A  is  used  to  denote  the  quantization  interval  length. 

The  quantization  of  the  coefficients  using  an  L-level  quantizer  provides  a  possi¬ 
ble  of  LAI  combinations  of  levels  for  each  coefficients  to  represent  the  vector,  which 
corresponds  to  LAI  possible  quantization  points.  Assuming  no  subsequent  entropy 
coding,  such  a  representation  requires  at  least  B  >  M  log2  L  bits  to  represent  the 
coefficients.  Alternatively,  given  a  fixed  budget  of  B  bits,  assuming  that  all  the  bits 
are  equally  distributed  to  all  the  coefficients,  then  the  quantizer  can  have  at  most 
L  <  2b/ai  levels  per  coefficient.  For  exposition  simplicity,  we  assume  that  B  and  L 
are  integers  such  that  L  =  2B^A1 . 
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Scalar  quantization  of  the  sampling  coefficients  partitions  RA/ — and,  correspond¬ 
ingly,  S  and  W — into  LM  cells  which  contain  the  quantization  points.  The  edges  of 
each  cell  in  Mm  and  W  are  orthogonal  to  each  other,  with  dimensions  that  are  implied 
by  the  quantization  intervals  of  the  scalar  quantizer.  The  quantization  levels  of  the 
quantizer  determine  the  quantization  points  inside  each  cell.  Any  vector  y  contained 
in  a  cell  is  quantized  to  the  quantization  point  y  corresponding  to  that  particular 
cell.  For  example,  the  cells  of  a  uniform  quantizer  are  all  hypercubes  of  size  AM ,  with 
the  corresponding  quantization  points  at  the  center  of  each  hypercube.  The  metric 
of  interest  is  the  i 2  norm  of  the  reconstruction  error  using  the  quantized  coefficients, 
i.e.  the  magnitude  ||x  —  R(y ) 1 1 2 -  A  two-dimensional  example  is  shown  in  Fig.  1(a). 

Depending  on  the  formulation,  the  performance  of  the  quantizer  is  commonly 
measured  using  either  the  expected  value  or  the  worst  case  quantization  error  over 
all  the  vectors  of  interest.  The  signal  is  assumed  to  lie  in  a  K -dimensional  subspace 
and,  without  loss  of  generality,  it  is  power-limited  by  |  |x|  1 2  <  1.  Under  these  assump¬ 
tions  the  minimum  worst-case  error,  and  the  minimum  mean  square  error  of  a  vector 
quantizer  can  be  derived  for  any  quantizer  that  uses  P  cells  to  partition  the  space. 
Using  the  minimax  approach  as  in  [5,  6]  the  worst  case  error  magnitude  can  be  shown 
to  be  greater  than: 

max  |[e|  |2  >  P~1/K ,  (3) 

X 

for  any  choice  of  vector  quantizer.  Similarly,  using  Zador’s  formula  [7, 8],  the  minimum 
mean  squared  error  for  any  quantizer  can  be  shown  to  be: 

E{\\e\\2}  =  n(P2/K),  (4) 

in  which  the  constants  only  depend  on  the  distribution  of  the  signal  in  the  subspace. 
Within  the  constant  of  proportionality,  both  results  are  essentially  the  same.  Thus, 
the  minimax  approach  is  used  in  the  remainder  of  this  paper,  ft  should  be  noted 
that  an  assumption  in  this  work,  consistent  with  the  minimax  approach,  is  that  the 
locations  of  the  sparse  coefficients  are  uniformly  distributed.  Often  correlations  in  the 
locations  of  the  sparse  coefficients  can  reduce  the  rates  required  to  encode  the  signals. 
In  such  cases,  more  general  methods,  such  as  the  information  theoretic  rate-distortion 
analysis  introduced  in  [9]  can  be  used  to  analyze  the  system  performance. 

2.2  Sparse  Representations  and  Compressive  Sensing 

Signals  of  interest  in  signal  processing  can  often  be  represented  or  well  approximated 
using  sparse  representations  in  a  particular  basis.  In  this  paper  it  is  assumed  that 
the  signals  of  interest,  x  G  S  are  all  the  K -sparse  signals  in  a  known  orthonormal 
basis  {0i, . . . ,  i/jn},  such  that  the  sum  in  the  representation  x  =  JA(x,  ipijipi  only 
has  K  non-zero  terms.  It  follows  that  the  signals  of  interest  lie  in  a  collection  of 
iF-dimensional  subspaces. 

Recent  advances  in  compressive  sensing  [1-4]  demonstrate  that  sparse  signals  can 
be  sampled  and  exactly  reconstructed  using  fewer  than  N  linear  measurements.  The 
signal  x  is  sampled  using  M  measurements  with  the  orthogonal  basis  {0i, . . . ,  0m}  to 
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Figure  1:  Example  of  uniform  scalar  quantization  in  a  two  dimensional  sampling 
space,  (a)  Any  vector  y  in  a  quantization  cell  is  quantized  to  the  center  of  the  cell  y. 
(b)  The  one-dimensional  subspace  TL  intersects  very  few  of  the  quantization  cells. 


produce  the  vector  y  =  <E>x,  as  described  in  Sec.  2.1.  It  is  possible  to  recover  x  using 
a  convex  program  if  the  sampling  operator  $  satisfies  a  restriced  isometry  property 
(RIP),  which  is  satisfied  with  very  high  probability  if  M  —  cK\og2(N/K)  random 
measurement  vectors  are  used  [4],  If  the  RIP  holds,  then  the  vector  x  that  satisfies: 

x  :=  min  ||x||^  s.t.  3>x  =  y,  (5) 

is  equal  to  the  original  vector  x.  Measurements  can  also  be  perturbed,  for  example 
due  to  measurement  noise  or  quantization,  by  an  error  vector  e,  in  which  case  the 
measured  signal  is  y  =  $x  +  e.  If  the  error  vector  is  bounded  by  ||e||^2  <  5  then  the 
signal  can  be  approximately  recovered  using  the  following  convex  program: 

x  :=  min  | |x| s.t.  j |3>x  —  y\\t2  <  5.  (6) 

In  this  case,  the  recovered  signal  x  satisfies  ||x  —  x||  <  C5  for  some  constant  C  that 
only  depends  on  the  sampling  operator  and  not  on  the  signal  x.  For  the  remainder  of 
this  paper  the  measurement  error  is  attributed  to  quantization.  The  performance  in 
the  presence  of  noise  can  only  be  worse.  A  study  of  the  performance  in  the  presence 
of  noise,  although  interesting,  is  beyond  the  scope  of  this  paper. 

3  Quantization  and  Compressive  Sensing 

This  section  examines  the  scalar  quantization  of  compressive  sensing  measurements. 
Section  3.1  presents  an  upper  bound  on  the  number  of  quantization  cells  intersected  in 
the  sampling  process.  Sections  3.2  and  3.3  use  the  bound  to  evaluate  the  rate  efficiency 
and  the  error  performance,  respectively.  Section  3.4  applies  the  error  bounds  to  the 
case  of  compressible  signals. 

3.1  Sampling  and  Quantization  of  Subspaces 

The  sampling  and  quantization  of  low  dimensional  signal  subspaces  using  higher  di¬ 
mensional  sampling  spaces  has  been  studied  in  the  context  of  frame  representations 
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and  oversampling  [6,8].  The  bounds  derived  in  both  references  are  similar,  but  the 
approach  in  [6]  can  be  immediately  applied  to  the  quantization  of  sparse  signals. 
Scalar  quantization  of  a  basis  expansion  in  an  M-dimensional  space  W  using  an  L- 
level  scalar  quantizer  generates  an  orthogonal  quantization  lattice  with  LM  vector 
quantization  cells.  It  is  shown  in  [6]  that  a  K- dimensional  subspace  of  W  intersects 
a  limited  number  of  cells,  denoted  using  /0(M,  K,  L),  which  is  upper  bounded  by: 

h(M,K,L)<(2L)K{^j.  (7) 

It  thus  follows  that  any  cells  outside  the  ones  intersected  are  never  used  by  the 
quantizer.  Figure  1(b)  presents  an  example  for  the  case  of  M  =  2  and  K  =  1,  in 
which  the  one-dimensional  subspace  7i  only  intersects  very  few  quantizations  cells. 
The  bound  in  Eq.  (7)  can  be  immediately  used  in  Eqs.  (3)  and  (4)  to  derive  a  lower 
bound  on  the  reconstruction  error  for  any  signal  in  the  subspace. 

In  the  case  of  sparse  signals,  this  result  is  immediately  applicable  by  realizing  that 
K -sparse  signals  occupy  a  set  of  (^)  subspaces  in  the  signal  space  S.  It  follows  that 
the  projection  implied  in  the  sampling  operation  produces  at  most  (^)  of  subspaces  of 
dimension  at  most  K  in  the  sampling  space  W.  Thus  the  number  of  cells  intersected, 
denoted  using  /^(M,  Ji,  L),  is  at  most: 

IN(M,K,L)<(2L)K(^j{^j.  (8) 

To  minimize  the  maximum  error,  these  quantization  levels  are  equally  distributed  to 
each  of  the  subspaces,  and  therefore  at  most  J0(M,  K,  L )  quantization  points  should 
be  used  to  encode  each  of  the  subspaces.  It  should  be  noted  that  this  is  equivalent 
to  the  assumption  that  the  K  components  of  the  sparse  signal  are  uniformly  chosen 
from  the  N  possible  components,  and  the  use  of  Zador’s  formula  in  Eq.  (4)  to  lower 
bound  the  achievable  mean  squared  error. 

3.2  Rate  Penalty  for  Compressive  Sensing 

A  consequence  of  the  discussion  above  is  that  there  is  a  rate  penalty  associated  with 
scalar  quantization  of  samples  of  sparse  signals.  This  section  quantifies  that  penalty 
and  examines  the  asymptotic  behavior  in  a  variety  of  asymptotic  assumptions. 

Assuming  a  fixed  number  of  quantization  levels  L,  the  rate  used  for  transmission 
is  at  least  B  =  Mlog2L.  On  the  other  hand,  only  In(M,  K,  L)  cells  are  intersected, 
which  implies  that  the  rate  required  is  at  most  log2(/(M,  K,  L)).  Thus,  an  upper 
bound  on  the  effective  bit  use  of  the  quantizer — which  is  also  a  measure  of  the  quan¬ 
tizer  efficiency — can  be  derived  from  Eq  (8): 

log 2(1N(M,K,L))  <  log2((2L)^QO)  <  A-log2(2Lf  f) 

M  log2  L  ~  M  log2  L  ~  M  log2  L 
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Specifically  in  the  case  of  compressive  sensing,  reconstruction  robustness  requires  that 
the  number  of  samples  is  equal  to  M  =  cK\og2(N/K)  for  some  constant  c  >  1.  In 
this  case,  the  bit  use  is  upper  bounded  by: 


\og2(IN(M,K,L))  <  log2L  +  log2(^)  +  log2(log2(r))  +  log2(e2c)  +  1 


M  log2  L 


dog2(^)log2L 


=  0 


C log2(f)  c log2  L 


(10) 

(11) 


As  the  quantization  becomes  finer,  L  grows  to  infinity  and  the  fraction  of  bits  used 
converges  to  l/(clog2(^)).  Similarly,  as  the  signal  becomes  sparser,  the  fraction 
p  =  N/K  grows  to  infinity,  and  the  fraction  of  bits  used  converges  to  l/(clog2  L). 

Alternatively,  the  bounds  above  could  be  derived  assuming  a  fixed  rate  of  B  bits, 
i.e.  L  =  2b/ai  quantization  levels.  In  this  case: 

log 2(In(M,K,L))  ^  K  ,  1  +  eK (log2  M  +  log2  N  —  2  log2  K) 

B  ~  M+  B  ' 

Again,  when  obtaining  M  =  cK  \og2{N / K)  compressive  sensing  measurements,  the 
bit  use  is  upper  bounded  by: 

log2(IN(M,K,L))  <  1  1  +  eA^(log2(| )  +  log2(clog2(f ))) 

B  ~  clog2(f)  B 

The  bounds  in  Eq.  (11)  and  (13)  describe  the  gap  between  the  number  of  bits 
used  by  the  quantized  representations  and  the  number  of  bits  that  are  necessary 
to  represent  the  same  quantization  points.  In  principle,  subsequent  lossless  entropy 
coding  could  be  used  to  eliminate  the  gap  between  the  bits  necessary  and  the  bits 
used.  However  such  a  coding  method  is  not  currently  available  for  compressive  sensing 
measurements.  Some  of  the  difficulty  in  implementing  a  lossless  entropy  coding  stage 
is  a  consequence  of  the  randomness  of  the  sampling  dictionary,  which  is  used  to 
achieve  the  universality  property.  The  gap  in  bit-use  efficiency  is  the  price  to  pay  for 
the  universality  of  the  currently  state  of  the  art  universal  encoding  strategies. 

3.3  Error  Bounds 

Coupled  with  the  rate  efficiency  of  a  sampling  and  quantization  algorithm  for  sparse 
representations  is  a  bound  on  the  maximum  decoding  error.  Specifically,  this  section 
uses  the  number  of  quantization  cells  intersected  by  the  sampling  method  together 
with  Eq.  (3)  to  determine  a  lower  bound  on  the  worst  case  quantization  error,  as¬ 
suming  the  best  possible  distribution  of  these  points  in  the  volume  of  interest.  That 
is  compared  to  the  best  case  lower  bound,  which  assumes  the  use  of  all  the  possible 
quantization  points  at  that  rate.  Furthermore,  the  efficiency  of  the  classical  com¬ 
pressive  sensing  paradigm  is  considered,  in  which  sampling  and  scalar  quantization  is 
followed  by  non-linear  reconstruction  using  linear  programming  algorithms. 
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As  a  benchmark  for  further  comparison,  a  lower  bound  on  the  worst  case  error 
is  derived  and  it  is  shown  that  it  is  achievable.  Specifically,  the  lowest  worst  case 
error  can  be  achieved  by  equally  distributing  the  available  quantization  points  to  all 
(^)  subspaces  that  constitute  the  signal  set.  Using  a  pre-determined  bit-rate  B ,  this 
corresponds  to  2s/  (^)  quantization  points  per  subspace.  From  Eq.  (3)  and  Stirling’s 
formula  it  follows  that  the  worst  case  error  is  lower  bounded  by: 


N 

eB  -  2 B/K  K ' 


(14) 


This  bound  is  tight  since  it  can  be  achieved  using  log2  (^)  ~  K  log2  ( j- )  bits  to  encode 
the  locations  of  the  sparse  coefficients  and  allocating  the  remaining  bits  to  encode 
the  implied  AT- dimensional  subspace  using  a  vector  quantizer,  ft  should  be  noted 
that  if,  instead,  a  scalar  quantizer  is  used  to  encode  each  of  the  K  sparse  coefficients 
independently,  there  is  a  K 1  /2  penalty  to  the  achievable  worst  case  error: 

N 

eB-Sc  A  2B/Kftl/2  • 


At  the  same  bit-rate  B,  sampling  followed  by  scalar  quantization  using  L  =  2B^AI 
levels  intersects  only  K,2B^M)  quantization  cells.  If  these  cells  and  the  cor¬ 

responding  quantization  points  are  efficiently  placed  in  the  space,  the  worst  case 
quantization  error  is  lower  bounded  by: 

«  S:  UM,  K,  2 Bn-1/K  >  21+B%Me-  <16) 

This  bound,  applied  to  M  =  cK  \og2(N /  K)  compressive  sensing  measurements,  yields 
the  following  lower  bound  for  any  reconstruction  algorithm: 


ecs  > 


2i+B/(c^iog2(iv/iC))celog2(A; 


=  n 


2B/(cKlog2(N/K))  log2(^ 


(17) 


Assuming  reconstruction  using  the  linear  program  in  Eq.  (6),  the  error  can  be  guar¬ 
anteed  to  be: 


£cs_i^o(M-A/2).0(Ah|lW),  (18) 

in  which  A  =  21~B^AI  is  the  uniform  quantization  interval  achievable  at  rate  B. 

The  lower  error  bound  in  Eq.  (17)  is  worse  than  the  lower  error  bound  in  Eq.  (14). 
The  logarithmic  term  in  the  denominator  of  the  exponent  can  be  explained  by  con¬ 
sidering  that  the  optimal  vector  quantization  method  only  needs  to  quantize  a  K- 
dimensional  space  instead  of  the  M  =  cK  log2(A/A")-dimensional  one  used  for  com¬ 
pressive  sensing.  On  the  other  hand  compressive  sensing  gains  an  N/K  factor  since 


the  location  of  the  sparse  components  does  not  need  to  be  encoded.  The  remain¬ 
ing  penalty  of  1/log 2(N/K)  should  be  attributed  to  the  inefficient  use  of  the  M- 
dimensional  space,  since  the  union  of  the  projected  subspaces  does  not  intersect  all 
the  quantization  cells,  ft  is  the  price  for  the  universality  of  compressive  sensing. 

There  is  a  further  gap  between  the  lower  bound  in  Eq.  (17)  and  the  achievable 
error  upper  bound  in  Eq.  (18).  A  factor  of  M1/2  =  (K  log2(N/  K))1^2  in  this  gap 
can  be  attributed  to  the  use  of  scalar  quantization  instead  of  vector  quantization  in 
the  M  dimensional  space.  However,  there  is  still  a  factor  of  log2(fV / K)  that  is  not 
accounted  for.  This  factor  is  due  to  the  inefficiency  of  the  recovery  process  and  the 
inefficient  placement  of  the  quantization  cells  on  the  subspaces.  A  similar  logarithmic 
gap  is  demonstrated  in  [5]  in  the  case  of  compressible  signal  models. 

3.4  Compressible  Signals 

As  mentioned  in  the  Introduction,  another  commonly  used  sparsity  model  is  the  set 
of  vectors  x  with  bounded  £p  norm  for  some  0  <  p  <  1.  This  section  uses  the  results 
developed  above  to  improve  on  the  bounds  presented  in  [5] ,  specifically  in  the  case  of 
low  bit-rates  and  highly  compressible  signals. 

Without  loss  of  generality  it  is  assumed  that  the  signals  lie  in  the  unit  ip  ball,  Bp  = 
|x|  1 1 x 1 1 ^ p  <  l}.  In  [5]  the  following  error  bound  is  presented  for  the  quantization  of 
such  signals  using  any  encoding  algorithm: 


=  n 


B 


log  2(N/B)  +  1 


-l/p+1/2 


(19) 


As  p  tends  to  0,  this  bound  also  tends  to  0  and  becomes  loose.  At  the  limit  of  p  =  0, 
it  can  be  shown  that  Bp  converges  to  the  set  of  signals  that  are  1-sparse1  and  power 
limited  to  1.  In  this  case,  Eq.  (14)  provides  a  tighter  bound  than  Eq.  (19): 


e4>^  =  2-B+log2JV  (20) 

Using  the  nesting  property  of  Pp  balls,  namely  that  Bp  C  Bp  if  p  <  p' ,  it  follows 
that  the  error  bound  is  non-decreasing  in  p.  Therefore,  Eq.  (20)  is  a  lower  bound  of 
the  quantization  error  for  all  p.  This  bound  is  tighter  than  the  one  in  Eq.  (19)  when 
p  is  small.  In  the  case  of  very  low  rate  quantization  (i.e.  B  close  to  log2  N)  the  bound 
in  Eq.  (20)  can  be  dominant  even  for  large  p  <  1.  However,  as  the  rate  B  increases, 
the  bound  decreases  exponentially  and  the  bound  in  Eq.  (19)  dominates,  even  for  p 
close  to  0. 


4  Conclusions 

This  work  demonstrates  that  compressive  sensing  of  sparse  signals  followed  by  scalar 
quantization  is  inefficient  both  in  terms  of  the  rate  utilization  and  the  error  per¬ 
formance.  Some  of  the  inefficiency  may  be  recovered  using  vector  instead  of  scalar 

1In  general,  limp_>o  {x|  ||xlbp  <  C }  is  the  set  of  1-sparse  signals  that  are  power  limited  to  \fC. 
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quantization.  Still,  the  universality  of  the  encoding  algorithm  requires  that  the  quan¬ 
tization  cells  uniformly  cover  the  sampling  space.  On  the  other  hand  the  samples  of 
a  sparse  signal  form  low  dimensional  subspaces  in  the  sampling  space.  These  sub¬ 
spaces  do  not  intersect  all  the  available  quantization  cells.  The  resulting  gap  is  the 
cost  of  universality.  The  results  suggest  some  further  inefficiency  in  reconstruction 
using  linear  programming.  This  might  be  recoverable  or  partly  recoverable  using 
reconstruction  methods  that  ensure  reconstruction  consistency  with  the  quantized 
measurements. 
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